Parallel Job Scheduling Policy for Workstation Cluster Environments

As workstation clusters (WC) become more commonly used for parallel jobs, there is a growing awareness for the need of job scheduling policies. There have been a fair number of studies on how to schedule parallel applications on parallel systems and a good survey in the area can be found in [5]. It has been shown that the best solution to the processor allocation problem in a distributed multiprocessor environment is an adaptive scheduling policy that can adjust load distribution based on runtime scheduling algorithms [2,3]. The main idea of adaptive space-sharing policies is that the number of processors assigned to a job is a compromise between the user’s request and what the system can provide. Note that there are differences in the architecture of the multiprocessor systems and WC-based distributed systems. For example, the processors in the multiprocessors systems are usually homogenous whereas those of WC are usually heterogeneous. This change of architectural environment requires important differences in the decisions made by the system scheduling policy. Most of adaptive scheduling policies for WC-based systems provide only rudimentary facilities for partitioning, i.e., space sharing, the processors among parallel jobs. In addition, parallel applications targeted to WC are typically resource-intensive, i.e. they require more resources than are available at a single site. However, existing adaptive scheduling policies cannot accommodate this requirement. This is because they may assign 1 processor to a job in the extreme cases [2] or lead to a processor fragmentation problem [3], i.e. groups of processors idle within a partition while some jobs may be waiting for free processors, which leads to low system utilisation and throughput.